1.1 Basic Information

Telco Systems is a company which have been working on design and development of high-performance network communications over 40 years. It is the global leader in telecommunications, providing excellent telecommunication service for customers. The service includes 5G internet, networking slicing, and more on. In our group, all of us have not tried to analyze such a company in telecommunication field.

1.2 Reason of Choosing Telco Systems

The reason of choosing this company is that Telco provides large amount of home phone service and internet service. These two stuffs are unavoidable in contemporary society. People get used to take mobile phones to go outside, and they adapt to such a life with internet. When people stay at home, they spend majority of times with interenet no matter working or playing. We can say that people cannot leave internet nowadays. Therefore, the company which is providing these services are pretty crucial. Telco is a good company, but it also has some drawbacks. Not all users satisfy with service, so we want to know in what place Telco can make an improvement. We care about how customers say about Telco, and what they dislike. We are here to help the company to find specific problems, to avoid unnecessary customers churn. Through analyzing the company, we wonder what factors Telco can improve to let customers have greater experiences of using. The best way to know the using experience is based on customers’ survey. Therefore, we choose a dataset about customers survey.

2.1 About Dataset

The Telco customer churn data contains information about Telco that provided home phone and Internet services to 7043 customers in California at the end of 2017 Quarter 3. Data includes customers’ basic information and it indicates which customers have left, stayed, or signed up for their service.
Studying such data can help companies identify the characteristics of lost customers, identify potential, soon-to-be-lost customers and develop appropriate strategies to retain them.
The dataset is WA_Fn-UseC_-Telco-Customer-Churn.csv. Before analyzing this dataset, we did some research about what churn represents, and why it is important to avoid churn in business. Churn in this dataset represents lost customers. Some people will be curious about why the company should spend time on retaining current customers or decreasing lost customers. In fact, acquire a new customer is much harder than retaining an existing customer. Company can pay for fewer price to retain existing customers rather than spend large amount of money on advertisement, and it is a profound strategy to maintain good reputation. Therefore, it is crucial to figure out current problems, and then to fix it up.

2.2 Variables

## 'data.frame':    7032 obs. of  21 variables:
##  $ customerID      : chr  "7590-VHVEG" "5575-GNVDE" "3668-QPYBK" "7795-CFOCW" ...
##  $ gender          : Factor w/ 2 levels "Female","Male": 1 2 2 2 1 1 2 1 1 2 ...
##  $ SeniorCitizen   : Factor w/ 2 levels "No","Yes": 1 1 1 1 1 1 1 1 1 1 ...
##  $ Partner         : Factor w/ 2 levels "No","Yes": 2 1 1 1 1 1 1 1 2 1 ...
##  $ Dependents      : Factor w/ 2 levels "No","Yes": 1 1 1 1 1 1 2 1 1 2 ...
##  $ tenure          : int  1 34 2 45 2 8 22 10 28 62 ...
##  $ PhoneService    : Factor w/ 2 levels "No","Yes": 1 2 2 1 2 2 2 1 2 2 ...
##  $ MultipleLines   : Factor w/ 3 levels "No","No phone service",..: 2 1 1 2 1 3 3 2 3 1 ...
##  $ InternetService : Factor w/ 3 levels "DSL","Fiber optic",..: 1 1 1 1 2 2 2 1 2 1 ...
##  $ OnlineSecurity  : Factor w/ 3 levels "No","No internet service",..: 1 3 3 3 1 1 1 3 1 3 ...
##  $ OnlineBackup    : Factor w/ 3 levels "No","No internet service",..: 3 1 3 1 1 1 3 1 1 3 ...
##  $ DeviceProtection: Factor w/ 3 levels "No","No internet service",..: 1 3 1 3 1 3 1 1 3 1 ...
##  $ TechSupport     : Factor w/ 3 levels "No","No internet service",..: 1 1 1 3 1 1 1 1 3 1 ...
##  $ StreamingTV     : Factor w/ 3 levels "No","No internet service",..: 1 1 1 1 1 3 3 1 3 1 ...
##  $ StreamingMovies : Factor w/ 3 levels "No","No internet service",..: 1 1 1 1 1 3 1 1 3 1 ...
##  $ Contract        : Factor w/ 3 levels "Month-to-month",..: 1 2 1 2 1 1 1 1 1 2 ...
##  $ PaperlessBilling: Factor w/ 2 levels "No","Yes": 2 1 2 1 2 2 2 1 2 1 ...
##  $ PaymentMethod   : Factor w/ 4 levels "Bank transfer (automatic)",..: 3 4 4 1 3 3 2 4 3 1 ...
##  $ MonthlyCharges  : num  29.9 57 53.9 42.3 70.7 ...
##  $ TotalCharges    : num  29.9 1889.5 108.2 1840.8 151.7 ...
##  $ Churn           : Factor w/ 2 levels "No","Yes": 1 1 2 1 2 2 1 1 2 1 ...
##  - attr(*, "na.action")= 'omit' Named int [1:11] 489 754 937 1083 1341 3332 3827 4381 5219 6671 ...
##   ..- attr(*, "names")= chr [1:11] "489" "754" "937" "1083" ...

• gender: Female or Male
• SeniorCitizen: customer is a senior citizen or not (Yes, No)
• Partner: customer has a partner or not (Yes, No)
• Dependents: customer has dependents or not (Yes, No)
• tenure: number of months the customer has stayed with the company
• PhoneService: customer has a phone service or not (Yes, No)
• MultipleLines: customer has multiple lines or not (Yes, No, No phone service)
• InternetService: customer’s internet service provider (DSL, Fiber optic, No)
• OnlineSecurity: customer has online security or not (Yes, No, No internet service)
• OnlineBackup: customer has online backup or not (Yes, No, No internet service)
• DeviceProtection: customer has device protection or not (Yes, No, No internet service)
• TechSupport: customer has tech support or not (Yes, No, No internet service)
• StreamingTV: customer has streaming TV or not (Yes, No, No internet service)
• StreamingMovies: customer has streaming movies or not (Yes, No, No internet service)
• Contract: contract term of the customer (Month-to-month, One year, Two year)
• PaperlessBilling: customer has paperless billing or not (Yes, No)
• PaymentMethod: Electronic check, Mailed check, Bank transfer (automatic), Credit card (automatic)
• MonthlyCharges: amount charged monthly
• TotalCharges: total amount charged
• Churn: customer churned or not (Yes or No)

For our exploratory data analysis, we did some preprocessing including cleaning up and converting. We dropped “NA” values from the dataset to simplify our analysis; we converted all variables into factor variables except tenure, MonthlyCharges, TotalCharges

3.1 Churn vs Not Churn

3.2 Chi-square Tests

We use \(\chi^2\) test to test if two categorical variables are independent base on contingency table.

## 
##  Pearson's Chi-squared test with Yates' continuity correction
## 
## data:  churn_vs_partner
## X-squared = 158, df = 1, p-value <2e-16

## 
##  Pearson's Chi-squared test with Yates' continuity correction
## 
## data:  churn_vs_phone
## X-squared = 0.9, df = 1, p-value = 0.3

## 
##  Pearson's Chi-squared test with Yates' continuity correction
## 
## data:  churn_vs_bill
## X-squared = 257, df = 1, p-value <2e-16

4.1 kDE plot of continuous variables

KDE plot is a Kernel Density Estimation Plot which depicts the probability density function of the continuous data variables. We can easily observe the distribution of samples with kde plot and when we want to compare the distributions of different samples, it won’t be affected by the samples’ size.

As we can see from tenure_kdeplot, customers with lower tenure are more likely to churn. And from MonthlyCharges_kdeplot, customers with higher monthlycharges are also more likely to churn. From TotalCharges_kdeplot, we can find that churn customers and left customers have very similar distributions. From these 3 kde plots, which means, tenure may be negatively correlated with customer churn rates and monthlycharges may be positively correlated with customer churn rates. Finally, totalcharges may only make a little attribution to customer churn rates.

4.2 Logistic regression

Logistic regression is the appropriate regression analysis to predict a binary outcome (the dependent variable) based on a set of independent variables.

To verify the conclusion we drew from kde plots numerically, we use the logistic regression model to classify churn with different features.

We can see from the anova test results.

## Analysis of Deviance Table
## 
## Model 1: Churn ~ tenure
## Model 2: Churn ~ tenure + MonthlyCharges
## Model 3: Churn ~ tenure + MonthlyCharges + TotalCharges
##   Resid. Df Resid. Dev Df Deviance Pr(>Chi)    
## 1      7030       7176                         
## 2      7029       6382  1      794   <2e-16 ***
## 3      7028       6376  1        6    0.017 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

model 2 is significantly better than model 1. However, model 3 is not under 99% significant level. Which means, model 3 may have not many improvements than model 2.

4.3 AUC and ROC Curve

We can use AUC and ROC to measure model 2 and model 3. AUC (Area Under The Curve) - ROC (Receiver Operating Characteristics) curve is a performance measurement for the classification problems at various threshold settings. ROC is a probability curve and AUC represents the degree or measure of separability. It tells how much the model is capable of distinguishing between classes. Higher the AUC, the better the model is at predicting 0 classes as 0 and 1 classes as 1. By analogy, the Higher the AUC, the better the model is at distinguishing between customer with left and churn.

We can compare the ROC curves of two models. The two ROC curves are almost same. And the AUC of model 2 is 0.808, which means if we randomly choose a churn customer and a left customer, the probability of ranking churn customer higher than left customer is 0.808. AUC of model 3 is 0.809.

Therefore, totalcharges only makes a little attribution to improve the performance of the classification model. In the 3 continuous variables, we can dismiss the influence of totalcharges to customer churn rates.

In the 3 continuous variables, tenure has negative coefficients with churn and monthlycharges has positive coefficients with churn. It means when we have a customer with lower tenure and high monthlycharges, he has more probabilities to churn. And totalcharges is not significant influence factor to customer churn rates in 3 continuous variables.

5.1 Simple correlations

Since most of variables are factors, it makes more sense to check their Spearman correlations.

Larger circle means higher correlation. We can see that churn has negative correlation with contract and tenure, which means that customer who stays longer with the company or has a longer contract terms is less likely to churn. Customer who signed up for online security service and has technical support plan is also less likely to churn. So it makes sense that contract and tech support have positive correlation, which means most customers who signed up for a technical support plan also have longer contract term.

5.2 Will Total Charge influence the churn rate with other variables?

First, tenure and totalcharge show a positive correlation. However, except for the time when tenure is less than 20, customer churn rate is higher, and the distribution of customer churn samples in other stages is more dispersed.

Second, for Multiplelines, the sample groups of No multiplelines and No phone service have higher churn rates when totalcharge is not high.

Finally, for the sample group of month-to-month contracts, customers are also prone to churn when the totalcharge is low.

5.3 Will Monthly Charge influence the churn rate with other variables?

First, the first two sample groups with the lowest MonthlyCharges had low customer churn and a high number of customers. However, as MonthlyCharges increased after that, customer churn also started to increase, mainly concentrated when TotalCharges were still low.

Second, the relationship between MultipleLines and MonthlyCharges leads to the following three conclusions.

1. If the customers have no multiple lines service, they are likely to churn if their monthly charge are greater than 75 or between 30 and 50.

2. If they have no phone service, we will find an interesting result that the more monthly charge they have, the less customer will churn.

3. And if the customers have multiple lines service, they are like to churn when they have a monthly charge approximately greater than 70.

5.4 Will Tenure influence the churn rate with other variables?

First, the customers who have no partner and short tenure are more likely to churn.

Second, the customers who have a month-to-month contract and short tenure are more likely to churn. Also, when they have long tenure and a 2-year contract, their possibility of churn is obviously increased.

Finally, the payment method is also a factor. The customers who use electronic check or mailed check are more likely to churn if their tenure is short.

5.5 Will Contract influence the churn rate with other variables?

Since the four services OnlineSecurity, OnlineBackup, TechSupport and DeviceProtection and the variable contract are categorical variables, a direct scatter plot can only see nine points in the two-dimensional plane. Therefore, we first numerate these categorical variables and then discretize their values (i.e., add random numbers to the numerical results so that the points are evenly dispersed in the two-dimensional plane).

These 4 variables (Online Security & Tech Support & Online Backup & Device Protection) are probably influence, because we find that when customer do not have anyone of them, they are more likely to churn if they have a month-to-month contract, except Device Protection. The result shows that there are many people have Device Protection and month-to-month contract but still churn, even though the amount of these customers is less than the one who have no Device Protection.